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DISCLAIMER 

This  report  and  those  involved  in  its  preparation  do  not  endorse  any 
product,  process,  or  company  stated  herein.  Use  of  these  means  by  anyone 
does  not  imply  certification  by  the  CALS  Test  Network. 
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1 .  Introduction 
1 . 1  Background 

The  DOD  Computer-aided  Acquisition  and  Logistic  Support  (CALS) 
Test  Network  (CTN)  is  conducting  tests  of  the  military  standard 
for  the  Automated  Interchange  of  Technical  Information,  MIL-STD- 
184 OA,  and  its  companion  suite  of  military  specifications.  The 
CTN  is  a  DOD-sponsored  confederation  of  voluntary  participants 
from  industry  and  government  managed  by  the  Air  Force  Logistics 
Command . 

The  primary  objective  of  the  CTN  is  to  evaluate  the  effectiveness 
of  the  CALS  standards  (Standards)  for  technical  data  interchange 
and  to  demonstrate  the  technical  capabilities  and  operational 
suitability  of  those  Standards.  Two  general  categories  of  tests 
are  performed  to  evaluate  the  Standards,  formal  and  informal". 
Formal  tests  are  large,  comprehensive  tests  that  follow  a  written 
test  plan,  require  specific  authorization  from  DOD,  and  may  take 
months  to  prepare,  execute,  and  report. 

Informal  tests  are  quick  and  short,  taking  only  a  few  hours  to 
set  up  and  execute.  They  are  used  by  the  CTN  technical  staff  to 
broaden  the  testing  base  by  including  representative  samples  of 
the  many  systems  and  applications  used  by  CTN  participants.  They 
also  allow  the  CTN  staff  to  gain  feedback  from  many  industry  and 
government  interpretations  of  the  Standards,  to  increase  the  base 
of  participation  in  the  CALS  initiative,  and  to  respond,  in  a 
timely  manner,  to  the  many  requests  for  help  that  come  from 
participants.  Participants  take  part  voluntarily  and  are  bene¬ 
fited  by  receiving  an  evaluation  of  their  latest  implementation 
(interpretation)  of  the  Standards,  interacting  with  the  CTN 
technical  staff,  gaining  experience  in  use  of  the  Standards,  and 
developing  increased  confidence  in  them.  The  results  of  informal 
tests  are  reported  in  Quick  Short  Test  Reports  (QSTRs)  that 
briefly  summarize  the  standard (s)  tested,  the  hardware  and  soft¬ 
ware  used,  the  nature  of  the  test,  and  the  results. 


1 


CTN  Test  Report 
91-023 


1.2  Purpose  of  the  Test  Analysis  Report 

This  report  involves  two  types  of  Department  of  Defense  (DoD)  engineering  data 

repositories:  the  Army's  Digital  Storage  and  Retrievel  Engineering  Data  System 

(DSREDS)  and  the  Air  Force's  Engineering  Data  Computer-Assisted  Retrievel  System 
(EDGARS) . 

The  Test  Analysis  Report  for  Task  Assignment  90-006  is  written  to  fulfill  the  following  objectives: 

a.  To  document  the  results  of  test  plan  execution  at  DSREDS  -  AVSCOM,  Sl  Louis,  MO. 

b.  To  gather  user  opinion  of  the  proposed  DSREDS  /EDGARS  site  digital  data  acceptance/ qualicy 
assurance  procedures. 

c.  To  provide  a  basis  for  corrective  and  follow-up  action  for  any  deficiencies  encountered. 

L  3  Organization  of  the  Test  Analysis  Report 

The  Test  Analysis  Report  is  divided  into  three  major  sections:  Section  1  states  the  purpose  of  this  report, 
provides  references  to  documents  relevant  to  this  report  and  identifies  the  test  participants;  Section  2  provides 
an  analysis  of  the  test  results  obtained  during  implementation  of  the  Test  Plan;  and  Section  3  summarizes  the 
demonstrated  capability  and  deficiencies  of  the  tested  DSREDS /EDGARS  site  digital  data  acceptance/quaiicy 
assurance  procedures  and  provides  a  list  of  refinements  that  are  to  be  incorporated  into  the  procedures. 

1*  4  Project  References 

The  following  documents  are  applicable: 

a.  Task  Execution  Plan  No.  90-006:  1990  Phase  II  Data  Acceptance  and  Quality  Assurance. 

b.  DSREDS  /ED  CARS  Site  -  Digital  Data  Acceptance /Quality  Assurance  Procedures;  9  February  1990 
(revised  30  March  1990) 

c  Test  Plan  -  Digital  Data  Acceptance/Quality  Assurance  Procedures:  23  February  1990 
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L  5  Test  Participants 

Accurate  -  M.  Sligh 
AVSCOM  -  E.  Beard 

I,  6  Test  Observers 

PM  CALS  -  R.  Mitchell 
Accurate  -  D.  White,  G.  Diamond 
AVSCOM  -  P.  Beherns,  D.  Phillips 
CTNO  -  F.  Tittle 

L  7  Test  Site 

The  DSREDS  site  visited  was: 

SITE  DATE 

AVSCOM,  SL  Louis,  MO  11  September  1990 
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Z  TEST  ANALYSIS 

The  tests  described  in  Section  4.2  of  the  Test  Plan  (see  Section  L3  (c)  of  this  report)  were  conducted  at  the 
AVSCOM  DSREDS  site.  The  tests  did  not  cover  the  two  new  procedures.  Procedure  1  -  Government  Contract 
Preparation  -  Digital  Engineering  Drawing  Data,  and  Procedure  2  -  Government  Conduct  Final  Reviews  - 
Engineering  Drawing  Package  (EDP),  that  were  added  in  the  30  March  1990  revision  of  the  DSREDS /ED  CARS 
Site  Digital  Data  Acceptance/Quality  Assurance  Procedures.  The  procedures  tests  begin  at  procedure  3.  The 
results  of  these  tests,  along  with  discussions  and  analyses,  are  presented  in  the  paragraphs  that  follow.  The 
results,  discussion  and  analyses  are  presented  on  a  procedure-by-procedure  basis. 

2.1  Test  of  DSREDS  /ED CARS  Site  Procedure  3 

The  purpose  of  this  test  was  to  validate  that  Procedure  3  of  the  DSREDS/EDCARS  Site  Digital  Data 
Acceptance/Quality  Assurance  Procedures  was  adequate  for  performing  physical  acceptance  of  magnetic  tape. 

2. LI  Results:  Test  of  DSREDS/EDCARS  Site  Procedure  3 

This  test  was  conducted  in  a  walkthrough  fashion.  The  Packing  Slip,  Data  Acceptance  Sheet,  Document 
Identifier  Report  and  MIL-STD-1840A  packaging  method  were  examined.  The  following  comments  were  made: 

a.  CAGE  codes  should  be  included  on  all  forms. 

b.  With  the  exception  of  the  signature  line  for  the  on-site  government  representative,  the  shipping 
list  currently  used  contains  the  information  proposed  in  the  Packing  Slip,  Data  Acceptance 
Sheet  and  Document  Identifier  Report 

c.  The  Packing  Slip,  Data  Acceptance  Sheet  and  Final  Acceptance  Sheets  should  be  merged  into 
one  document 

d.  The  Document  Identifier  Report  should  be  automated.  The  preferred  media  would  be  9-track 
tape. 
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2*1.2  Discussion  and  Analysis:  Test  of  DSREDS/EDCARS  Site  Procedure  3 

In  response  to  comments  (a),  (b)  and  (c),  a  new  document  that  incorporates  the  information  found  in  the 
Packing  Slip,  Data  Acceptance  Sheet  and  Final  Acceptance  Sheet  should  be  developed  in  order  to  reduce  the 
number  of  reports  that  must  be  handled.  It  is  suggested  that  the  Phase  II  procedures  implement  these  suggested 
changes. 

In  response  to  comment  (d),  delivering  the  data  found  in  the  Document  Identifier  Report  in  digital  form  (9- 
track  tape)  would  case  automating  some  of  the  quality  assurance  procedures  and  should  be  considered  for 
implementation  within  the  computer-assisted  Phase  II  Data  Acceptance  procedures. 

2*2  Test  of  DSREDS/EDCARS  Site  Procedure  4 

The  purpose  of  this  test  was  to  validate  that  Procedure  4  of  the  DSREDS/EDCARS  Site  Digital  Data 
Acceptance/Quality  Assurance  Procedures  is  adequate  for  performing  conversion  of  incoming  184GA/28002  tape 
to  DSREDS/EDCARS  format. 

2*2»1  Results:  Test  of  DSREDS/EDCARS  Site  Procedure  4 

This  procedure  is  not  currently  in  use  because  the  means  of  performing  l84QA/28Q02-to-DSREDS/EDCARS 
Tape  format  conversion  is  not  currently  available.  However,  the  following  comment  was  made:  the  conversion 
report  could  be  compared  electronically  with  the  Document  Identifier  Report  and  exceptions  reported.  The 
proposed  Conversion  Report  could  be  combined  with  the  current  ISR  26000  report. 
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222  Discussion  and  Analysis:  Test  of  DSREDS/EDCARS  Site  Procedure  4 

Once  the  method  of  conversion  is  established,  methods  for  electronically  comparing  the  conversion  report  with 
the  Document  Identified  Report  should  be  developed. 

23  Test  of  DSREDS/EDCARS  Site  Procedure  5 

The  purpose  of  this  test  is  to  validate  that  Procedure  5  of  the  DSREDS/EDCARS  Site  Digital  Data 
Acceptance/Quality  Assurance  Procedures  is  adequate  for  performing  population  of  DSREDS/EDCARS 
databases. 

23.1  Results:  Test  of  DSREDS/EDCARS  Site  Procedure  5 

It  was  suggested  that  paragraph  43(d)  of  the  DSREDS/EDCARS  Site  Digital  DA/QA  Procedures  be  deleted. 
That  paragraph  addressed  the  preparation  of  a  Population  Acceptance  Sheet. 

2 32  Discussion  and  Analysis:  Test  of  DSREDS/EDCARS  Site  Procedure  5 

Since  the  Database  Population  Report  could  be  compared  electronically  with  the  Document  Identifier  Report, 
and  the  exceptions  reported,  the  need  to  produce  the  Population  Acceptance  Sheet  for  manual  inspection  is  no 
longer  needed. 

24  Test  of  DSREDS/EDCARS  Site  Procedure  6 

The  purpose  of  this  test  is  to  validate  that  Procedure  6  of  the  DSREDS/EDCARS  Site  Procedures  is  adequate 
for  performing  visual  validation  of  image  and  key  identification  data. 
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2.4.1  Results:  Test  of  DSREDS/EDCARS  Site  Procedure  6 

It  was  suggested  that  paragraphs  4.6.2  (c),  (d)  and  (e)  of  the  DSREDS/EDCARS  Site  Digital  DA/QA 
Procedures  be  deleted.  These  paragraphs  addressed  border  existence  image  skew  and  line  separation. 

2.4.2  Discussion  and  Analysis:  Test  of  DSREDS/EDCARS  Site  Procedure  4 

Paragraphs  4.6.2  (c)  and  (d)  should  be  deleted  because  a  drawing  could  be  both  legible  and  reproducible  and 
yet  be  skewed  on  without  a  border.  Paragraph  4.6 2  (e)  is  redundant  because  it  is  covered  in  4.6.2  (b). 

2 *5  Test  of  DSREDS/EDCARS  Site  Procedure  7 

The  purpose  of  this  test  is  to  validate  that  Procedure  7  of  the  DSREDS/EDCARS  Site  Digital  Data 
Acceptance/Quality  Assurance  Procedures  is  adequate  for  performing  final  acceptance. 

2*5.1  Results:  Test  of  DSREDS/EDCARS  Site  Procedure  7 
The  following  comments  were  made: 

a.  The  letter  referred  to  in  paragraph  4.7  should  be  a  ’contract  letter*. 

b.  The  Final  Acceptance  Sheet  should  be  merged  with  the  Packing  Slip  and  Data  Acceptance 
Sheet 

252  Discussion  and  Analysis:  Test  of  DSREDS/EDCARS  Site  Procedure  7 

In  response  to  comment  (a),  paragraph  4.7  should  be  revised  so  that  'letter'  is  replaced  with  ’contract  letter* 
because  the  Contracting  Officer  is  the  final  authority  for  acceptance. 

In  response  to  comment  (b),  the  Final  Acceptance  Sheet,  Packing  Slip  and  Data  Acceptance  Sheet  should  be 
merged  into  a  document.  This  single  document  would  then  track  the  digital  data  from  the  loading  dock  to  final 
acceptance  at  the  receiving  site. 
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3.  SUMMARY  AND  CONCLUSIONS 

The  five  (5)  principal  procedures  were  tested  in  depth,  using  the  test  plan  to  record  the  results  and  the  test 
procedures  in  conducting  the  testing.  The  testing  went  very  well  and  in  general,  the  procedures  were  found  to 
be  acceptable.  A  few  key  recommendations  were  made  that  integrated  some  of  the  forms  to  reduce  the  paper 
required  to  record  the  acceptance  of  the  data.  It  was  recommended  that: 

L  The  Packing  List  and  Data  Acceptance  Sheet  be  combined  and  expanded  in  to  include  space 
for  final  acceptance  sign  offs  thereby  eliminating  the  final  acceptance  sheet  also. 

2.  The  Conversion  Acceptance  Sheet  and  the  Database  Population  Acceptance  Sheet  be  eliminated 
if  and  when  the  comparisons  are  made  electronically.  (Computer  Assisted  techniques) 

3.  The  Document  Identifier  Report  "batch  data"  be  delivered  on  9-track  magnetic  tape,  loaded 
into  the  system  when  batch  data  is  converted  that  the  key  data  fields  be  compared  within 
the  computer  and  a  computer  report  be  prepared  showing  the  results. 

4.  When  the  TEF  Database  Population  Report  (DSREDS/EDCARS  ISR  26000)  is  prepared,  it 
will  show  any  difference  between  it  and  the  delivered  Magnetic  Tape  batch  identification  "Key 
Field"  data  if  it  exists. 

These  suggestions  are  presently  being  considered  for  incorporation  into  the  Phase  II  Computer  Assisted  DA 
Procedures.  Additional  comments  were  made  by  AVSCOM  that  related  to  the  test  plan  steps  but  did  not  impact 
the  procedures. 
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